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ABSTRACT 

A survey was conducted of Colorado postsecondary 
career and technical programs in 1995 to gauge how the state has 
responded to mandates of the Carl Perkins Vocational and Applied 
Technology Act of 1990. Results were reported for the first three of 
nine standards developed by a statewide committee in 1992. The three 
standards require postsecondary programs to demonstrate student 
achievement in career and technical programs in the following areas: 
(i) basic and applied academic skills; (2) general occupational 
skills; and (3) specific occupational skills. Information was 
gathered through a survey questionnaire mailed to each postsecondary 
provider, with a follow-up, resulting in returns from 477 programs 
(91 percent) . In the area of standard one, the survey results showed 
that programs use both commercial instruments . (52X) and locally 
developed instruments (34%) to measure attainment in this area. Fewer 
than one-quarter of the programs had results available to show either 
gain scores or other data for standard one. For standard two, 
programs use a variety of methods to assess general occupational 
skills, with most programs using more than one method of assessment. 
The most frequently occurring method is tests, followed by 
checklists, "other," portfolios, and skills profiles. Nearly 19 
percent of postsecondary programs indicate that they use pre- and 
posttests to assess general occupational skills whereas the remainder 
use other methods. About 38 percent of the programs were able to 
document student general occupational skills attainment. For standard 
three, programs use a variety of methods to assess specific 
occupational skills. The most frequent method is locally-developed 
assessments, followed by successful completion of the program, 
certification examinations, commercial instruments, portfolios, and 
"other." Nearly 22 percent of postsecondary programs indicate that 
they use pro“ and posttests to assess specif.ic occupational skills 
whereas the remainder use other methods. About 44 percent of the 
progi ams had results available to document student specific 
occupational skills attainment. (KC) 
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results reported here. 

Thp «;ijrvev Questionnaire was mailed to each postsecondaiy provider in 
May 199s!^ A follow-up mailing to non-respondents jailed in Augus 
1 995 COCOES staft was involved in personally ^ 

espondents. By November 1995, 477 programs had reigned 
questionnaires, representing 90.7% of ail postsecondary career and 
technical programs in Colorado. 
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How have institutions responded to Standard One? 



Standard One deals with demonstrating 
student gains in basic and applied 
academic skills. This standard deals with 
how well programs integrate basic skills 
within the career and technical 
curriculum. LoQt' 

Programs use both commercial (52%) 

and locally-developed (34%) instruments ’ 

or assessments to measure student 

attainment in this area. About 14 percent 

of programs report they use "other" 

sources of measurement to determine 

gains in basic and applied academic 

skills, a finding that is puzzling given 

that these programs did not specify 

alternative methods. 



Sources of Measurement 
Basic and Applied Academic Skills 






Samples v.AII Students 
Basic and Applied Academic Skills 



Programs also were asked whether they 
measured basic and applied academic 
skills of all students or samples of 
students. Responses suggest that 
postsecondary programs are almost 
evenly split in choice of measurement 
technique. 



Samples (48 82%)- 
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Availability of Results 
Ra«5ir snd Anoiied Academic Skills 



According to this survey, less than 
one-quarter of postsecondary 
programs have results, either gain 
scores or other data, to report for 
Standard One. Results were most 
frequently available for mathematics, 
followed by .English, and reading. 
About 30 programs reported the 
availability of results in areas "other" 
than traditional basic skills. 
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There are several techniques for 
demonstrating gain in basic and 
applied academic skills. These 
include obtaining data from 
program entrants and comparing 
these data to program completers. 

Alternatively, some programs test 
the same students' at entrance and 
at exit to produce gain scores. 

About 28 percent of Colorado 
postsecondary career and 
technical programs test basic 
skills functioning of the same students or a sample of the same students at 
entrance and at exit. The remainder of programs report that they compare data 
in order to approximate gains. 




How have institutions responde d to Standard Two? 



Standard Two deals with demonstrating student gains in general occupational 
skills. General occupational skills are those skills needed for success in any 
workplace, no matter what career is chosen. 



Programs use a variety of methods 
to assess general occupational 
skills. Programs frequently use 
multiple methods of assessment. 

The most frequently occurring 
method is tests, followed by 
checklists, "other," portfolios, and 
skills profiles. The category "other" 
may include demonstrations or other 
forms of rating devices. 



Sources of Measurement 
General Occupational Skills 

Olhtr 
Portfolios 
local PorfornuuKe 
Local Written 
Slant Pronics 
Cbccklitts 
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Nearly 19 percent of postsecondary 
programs indicate that they use pre- 
and post-tests to assess general 
occupational skills while the remainder 
use other methods. Unlike basic and 
applied academic skills (Standard 
One), this standard does not require 
that students demonstrate a gain. 
Nonetheless, many postsecondary 
programs suggest those gain scores 
for general occupational skills are in 
use. 



Pre- and Post-testing 
General Occupational Skills 




Availability of Results 
General Occupational Skills 



Programs were asked to specify whether 
results were available to document 
student general occupational skills 
attainment. Almost 38 percent of the 
programs indicate availability of results. 
Such results are required for program 
approvals granted after July 1, 1995. 




How have institutions responded to Standard Three ? 

Sources of Measurement 



Programs use a variety of 
methods to assess specific 
occupational skills. The 
most frequently occurring 
method is locally developed 
assessments (both written 
and performance), followed 
by successful completion (of 
the program), certification 
examinations, commercial 
instruments, portfolios, and 
"other." 



Specific Occupational Skiils 



Portfolios 
Commerdil 
Loci] Performince 
LociJ 
Certification Eiuns 
Successful Complttlon 
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Nearly 22 percent of postsecondary 
programs indicate that they use pre- and 
post-tests to assess specific 
occupational skills while the remainder 
use other methods. Unlike basic and 
applied academic skills (Standard One), 
neither general or specific occupational 
skills standards require that students 
show a gain. Nonetheless, many 
postsecondary programs suggest that 
they produce gain scores for general and 
specific occupational skills. 



Pre- and Post-testing 
Specific Occupational Skills 




Programs were asked to specify 
whether results were available to 
document student specific 
occupational skills attainment. 
Almost 44 percent of the programs 
responding indicated availability of 
results. Such results in this area 
are required for program 
approvals granted after July 1, 
1995. 



Availability of Results 
Specific Occupational Skills 
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Colorado Postsecondary Vocational Standards, Survey Results 



Standard One-Basic and Applied Academic Skills 

Sources of Measurement for Basic and Applied Academic Skills 



% of total 



14% 


66 


13% 


64 


12% 


58 


22% 


105 


0% 


1 


0% 


1 


0% 


1 


0% 


2 


0% 


1 


17% 


82 


4% 


20 


24% 


113 


19% 


91 


12% 


58 


23% 


110 



American College Testing (ACT) 

American College Testing (ACT)AWork Keys 
ASSET (ACT) 

California Achievement Test (CAT) 

Comprehensive/California Test of Basic Skills (CTBS) 

CAP Mathematics Skills Test 

College Assessment of Academic Proficiency (CAAP) 

College Basic Academic Subjects Examination (College BASE) 
Scholastic Achievement Test (SAT) 

Test of Adult Basic Education (TABE) 

Vocational Materials Laboratory, Academic Skills Portion 



mmw 



Checklist of basic and applied academic skills 
Test of basic and applied academic skills 
Student portfolio demonstrating basic skills proficiency 



Measurement Strategies for Basic and Applied Academic Skills 



% of total 



42% 


199 


29% 


137 


28% 


134 


18% 


85 


21% 


98 


21% 


99 


25% 


117 



All students at program entr^* 

All students at program exit 
A sample of students at program entry 
A samp le of students at program exit 



All students at program entry and exit 
Samples of students at program entry and exit 



Availability of Results for Basic and Applied Academic Skills 

% of total n 



9% 


41 


9% 


42 


7% 


35 


1% 


7 


9% 


42 


10% 


48 


13% 


61 


11% 


54 


4% 


20 


26% 


126 



Achieving English scores of [n],or better 
Achieving mathematics scores of [n] or better 
Achieving reading scores of [n] or better 
Achieving [other academic area] scores of [n] or better 
Other results for program completers 



Achieving English scores of [n] or better 
Achieving mathematics scores of [n] or better 
Achieving reading scores of [n] or better 
Achieving [other academic area] scores of [n] or better 
Other results for program enrollees 
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Colorado Postsecondary Vocational Standards, Survey Results 

Standard Two-Genera! Occupational Skills 

Sources of Measurement for Genera! Occupational Skills 

% of total n 



Locally-developed 
Standardized 



49% 


233 


2% 


10 


10% 


50 


51% 


242 


58% 


276 


9% 


44 


26% 


122 



'iali 






Written 

Performance 



Measurement Strategies for General Occupational Skills 

% of total n 



fp/sataocc 



7% 


35 


40% 


190 


9% 


43 


11% 


53 


18% 


87 


1% 


3 


30% 


145 



All students at program entry 
All students at program exit 
A sample of students at program entry 
A sample of students at program exit 

m ' 

All students at program entry and exit 
Samples of students at program entry and exit 



Average Results for Genera! Occupational Skills 



Avg. 

94.9 

98.7 



% achieving [84.21% of general occu pational skills 
% achieving [70.0]% of general occupational skills 
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Standard ThreQ-Specific Occupational Skills 

t 3ourc6s of Msftsuromsnt for Spscific OccupstionsI Skills 



% of total n 



25% 


118 


41% 


194 


1% 


7 


59% 


282 


51% 


243 


12% 


55 


3% 


14 


24% 


114 


40% 


192 


7% 


34 


»asurement 


of total 


n 


16% 


78 


44% 


208 


9% 


43 


10% 


49 


18% 


84 


4% 


20 


28% 


135 



Locally-developed 
Standardized 



Written 
Performa nce 






Written 

Performance 



Strategies for Specifio Occupational Skills 



All students at program entry 
All students at program exit 
A sample of students at program entry 
A sample of students at program e>dt 






All students at program entry and exit 
Samples of students at program entry and exit 



Average Results for Specific Occupational Skills 



Avg. 

97.3 % achieving [ 81 .8]% of general occupational skills 

89.6 % achieving [n] score on certification/licensure examinations 



98.6 % achieving [78.3]% of general occupational skills . 



NOTE; Percentages may not sum to 100% because of multiple survey responses 
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